5 Takeaways & An Overview of FDA's Guidance on Assessing Electronic Health Records and Medical Claims Data for Regulatory Decision-Making
5 Takeaways & An Overview of FDA's Guidance on Assessing Electronic Health Records and Medical Claims Data for Regulatory Decision-Making
Introduction
FDA released the final guidance, Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making, on July 25, 2024. This work follows the Framework for FDA’s Real-World Evidence (RWE) Program released in 2018 and 5 additional guidances concerning Real-World Data (RWD) released subsequently in the wake of the 2016 21st Century Cures Act which called for the FDA to explore the potential of RWD in regulatory decision-making. Three centers within the FDA developed the guidance: the Center for Drug Evaluation and Research (CDER), the Center for Biologics Evaluation and Research (CBER), and the Oncology Center of Excellence (OCE), to clarify the agency’s thinking on the use of RWD coming electronic health records (EHRs) and medical claims in clinical studies to support new drug and biologic approvals.
Background
FDA has used RWD for post-market surveillance for some time and RWD has been used sparingly for new drug approvals and expanded indications. The agency does not require randomized clinical trials (RTCs) to support drug approval, it requires “adequate and well-controlled clinical investigation” as codified in Code of Federal Regulations (CRF), Title 21, Chapter I, Subchapter D, Part 314. OMP’s Dr. John Concato, speaking at a Regan Udall RWD/RWE Guidance Webinar Series meeting, noted that FDA staff aren’t required to memorize this section of the federal code but, refer to it so often it becomes part of memory.
About the Guidance
The guidance spans 39 pages and covers a broad range of considerations, too extensive to be addressed in a single blog post. Some of the content mirrors discussions from FDA’s 2013 Guidance for Industry and FDA Staff: Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data, which was developed to fulfill Prescription Drug User Fee Act IV (PDUFA IV) commitments.
FDA does not endorse any specific data sources, study methodologies, or data standards in this guidance. Instead, it offers considerations for conducting adequate and well-controlled clinical investigations using data from EHRs and claims data. Key areas of focus include data sources, data capture, missing data, and validation.
The guidance starts by noting that sponsors should submit protocols and statistical analysis plans before conducting an RWD study. All essential elements of study design, analysis, conduct, and reporting should be predefined as required RCTs. The FDA isn’t reinventing the wheel with RWD but adapting it so the ride will be safe and effective on a broader range of terrains.
Data Sources
Given the current variability of RWD, including EHRs and claims, each data source proposed for a study needs to be evaluated within the context of the study question. The appropriateness of a data source for a particular study depends on its containing the right data elements over a sufficient length of time to answer the study question. For example, understanding whether a data source from a given health system comprehensively captures a patient’s care is necessary to determine whether the source includes all relevant outcomes.
Key Study Variables
The selection, definition, and validation of key study variables are critical elements of a RWD study. Key study variables include study population inclusion and exclusion criteria, exposure, outcome, and covariates. The guidance proposes 2 sets of definitions be developed for key study variables, a conceptual definition, and an operational definition. The conceptual definition “should reflect current medical and scientific thinking regarding the variable of interest”. The operational definition consists of the data element(s) and value(s) in the RWD used to determine if the conceptual definition is satisfied. It is recommended that all definitions be included in the study protocol and that the potential misclassification of all key study variables be evaluated along with the possible impact on study findings.
Missing data is a common challenge in the use of RWD and the extent of missingness for key variables should be assessed. It is important to understand if the missing data was intended to be collected in the source data or if the data elements are absent because they are not typically collected in the type of real world data being used. For example, blood pressure isn’t collected in medical claims, but a claims code can indicate high blood pressure. Sponsors need to understand the reasons for the absence of information in each RWD source and evaluate if it is it missing at random or if it could introduce bias into the study. The understanding and assumptions around missing data should be reflected in the study protocol and the statistical analysis plan.
Study Design Elements
The FDA is aware of the risk of multiple testing in the data by the sponsor with only the data yielding the desired results being submitted. The guidance notes that “[t]he study questions of interest should be established first, and then the data source and study design most appropriate for addressing these questions should be determined.” However, the boundary between analysis to determine if a data source is fit for purpose and conducting the study itself is still a bit blurry.
Changes in standard of care, treatment methods, coding conventions, and coding versions over time can affect study results. The guidance notes the importance of considering the time period from which subjects are selected from an RWD source to understand any potential impacts on the fitness of the data source for the study.
Data Quality & the RWD Lifecycle
RWD used in clinical studies undergoes several stages, starting with data accrual in the source. A subset of that source data will be extracted, standardized, and then stored for use. The FDA refers to this process as curation. The curated data may then be converted into another format or structure (defined by FDA as transformation) and, if needed, de-identified before being used in a study-specific dataset for analysis. The guidance specifies that sponsors should validate that there is no loss of information in this process. However, the FDA does not currently endorse any set of guidelines. Some may remember a 3rd party initially created the original CDISC data checks before the FDA took control of the work. A similar process may occur for RWD.
The guidance includes the expectation of documentation of the data management process and quality management for the lifecycle of the RWD as it moves to becoming RWE. This documentation should consist of a QA/QC Plan. These data and quality management processes are established for RCTs but have yet to be standardized for RWD, something that will likely evolve with time and as the FDA and sponsors gain more experience with RWD.
Conclusion
This guidance marks a step forward in integrating EHR and claims data into the regulatory framework of drug and biologic approvals. While it provides important considerations for conducting RWD studies, the guidance underscores the importance of maintaining the basic elements of good clinical practice. By addressing considerations around the use and analysis of EHR and claims data to support a clinical study, this guidance provides a set of factors that can be used to evaluate RWD data sources to conduct studies that can be effectively and reliably used to support regulatory decisions. By taking this approach instead of being prescriptive, the agency leaves room for more exploration and innovation with RWD. As the use of RWD in clinical studies continues to evolve, ongoing dialogue between the FDA, sponsors, and other stakeholders will be essential in refining these processes and ensuring the safety and efficacy of new therapies.
5 Key Takeaways
1. The standard of adequate and well-controlled clinical investigations needed for drug approval (CFR 21 Part 314) also applies to RWD studies.
2. The guidance does not endorse specific data sources, study methodologies, or standards. Instead, it provides insight into the FDA’s concerns and considerations so sponsors can address these when conducting studies using data from EHRs and medical claims. This is a balance between FDA ensuring studies conducted in RWD can be evaluated for drug approval without stifling innovation in the use of this data by being overly prescriptive.
3. Pre-specification is important. The balance of how much investigation can be done in the source data to determine fit for purpose before defining key study variables is still undefined. The FDA is aware of the risk of multiple testing by the sponsor with only the results yielding data set being submitted.
4. The guidance proposes 2 sets of definitions be developed for key study variables, a conceptual definition, and an operational definition and these be documented in the protocol. Key study variables include study population inclusion and exclusion criteria, exposure, outcome, and covariates.
5. RWD should be characterized for the completeness, conformance, and plausibility of data values. Work is required to ensure RWD is not altered meaningfully from the source data to the final data set analyzed and ultimately submitted to the FDA. The submission of documentation of the Data Management Process and Quality Management, including documentation of the Quality Assurance (QA) and Quality Control (QC) Plan.